Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

switch pipelines and remote resolver controllers on prod m01 to state… #5398

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

jkhelil
Copy link
Contributor

@jkhelil jkhelil commented Jan 27, 2025

…fulSets

linked to redhat-appstudio/o11y#462

@openshift-ci openshift-ci bot requested review from enarha and ramessesii2 January 27, 2025 14:31
Copy link

openshift-ci bot commented Jan 27, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: jkhelil
Once this PR has been reviewed and has the lgtm label, please assign roming22 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jkhelil
Copy link
Contributor Author

jkhelil commented Jan 27, 2025

@hugares @enarha can you have a look ?

Copy link
Contributor

@enarha enarha left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Strictly speaking that configuration slightly differs from the one in staging where tekton-pipelines-remote-resolvers is missing from the list of statefulSets. It is still working though as I see the statefulSet exist on the cluster. Ideally we'll have matching configurations in staging and prod. I do not mind which changes as far as they match. (I know that because I'm working on the same change, but you beat me to it).

spec:
containers:
- args:
- --threads-per-controller=32
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is that really needed? I'm looking at the existing deployment on m01 and I see the following parameters passed to the container -kube-api-burst=50 -kube-api-qps=50 -threads-per-controller=32 --threads-per-controller=32 . Isn't enough we have threads-per-controller in the performance field?

Copy link
Contributor Author

@jkhelil jkhelil Jan 27, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As you can see, I moved the sections for tekton-pipelines-controller and tekton-pipelines-remote-resolvers under spec/pipeline/statefulSets. This ensures that the topologySpreadConstraints, as well as resources/limits and requests, are applied to both controllers.

The threads-per-controller=32 setting should now be superseded by the performance flags defined under spec/pipeline/performance.

On the staging environment, we already have the same configuration for tekton-pipelines-controller. However, at the time I made this change, there was nothing related to remote-resolvers under spec/pipeline/deployments, as this section had already been moved to spec/pipeline/statefulSets

the change is conform to the existant, and yes it is slightly different from stage (for remote-resolvers, because i guess on stage we didnt configure remote-resolvers limits and requests)

if you are trying to align stage with prod, lets do your PR before this then, we can hold this for a bit

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The threads-per-controller=32 setting should now be superseded by the performance flags defined under spec/pipeline/performance.

That was my point, given that threads-per-controller=32 is already set through spec/pipeline/performance and it is indeed passed to the container, we do not need it here.

On the staging environment, we already have the same configuration for tekton-pipelines-controller. However, at the time I made this change, there was nothing related to remote-resolvers under spec/pipeline/deployments, as this section had already been moved to spec/pipeline/statefulSets

I see here https://github.com/redhat-appstudio/infra-deployments/pull/5036/files#diff-e961bdc3a405dfda1b39ed17ee82576bb68e2a35e9288e7de208e695e4ee5944L2226-L2228 is removed from deployments and not added to statefulSets. This change includes it. If we indeed need it, then we should update the staging overlay accordingly. I do not want to mix prod and staging changes, so I can open a separate PR to update staging. Once we deploy and validate it, we can proceed with this change.

if you are trying to align stage with prod, lets do your PR before this then, we can hold this for a bit

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@enarha sound good to me, I wil wait for your PR on stage, once merged, we proceed with this one

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK, I'll open one in a min. But what about the --threads-per-controller=32 (my original comment). Do you agree it should be removed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I will removed it

@jkhelil
Copy link
Contributor Author

jkhelil commented Jan 30, 2025

/hold
we need to discuss monitoring updates

@jkhelil jkhelil force-pushed the update_prod_m01_sts branch 2 times, most recently from db8a8f0 to 9b905d4 Compare January 30, 2025 08:27
@jkhelil jkhelil force-pushed the update_prod_m01_sts branch from 9b905d4 to d131e60 Compare January 30, 2025 08:28
@jkhelil
Copy link
Contributor Author

jkhelil commented Jan 30, 2025

/unhold

@jkhelil
Copy link
Contributor Author

jkhelil commented Jan 30, 2025

/hold

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants